Bayesian Sparse Unsupervised Learning for Probit Models of Binary Data

We present a unified approach to unsupervised Bayesian learning of factor models for binary data with binary and spike-and-slab latent factors. We introduce a non-negative constraint in the spike-and-slab prior that eliminates the usual sign ambiguity present in factor models and lowers the generalization error on the datasets tested here. For the generative models we use probit functions, which can be sampled without tuning parameters, unlike previous works that used logistic functions. The posterior distributions involve mixtures of binary and truncated Gaussian variables, for which we present exact Hamiltonian Monte Carlo samplers and compare their properties to Gibbs samplers.